A GPU Register File using Static Data Compression

GPUs rely on large register files to unlock thread-level parallelism for high throughput. Unfortunately, large register files are power hungry, making it important to seek for new approaches to improve their utilization. This paper introduces a new register file organization for efficient register-packing of narrow integer and floating-point operands designed to leverage on advances in static analysis. We show that the hardware/software co-designed register file organization yields a performance improvement of up to 79%, and 18.6%, on average, at a modest output-quality degradation.Comment: Accepted to ICPP'2

PERF: Performant, Explicit Radiance Fields

We present a novel way of approaching image-based 3D reconstruction based on radiance fields. The problem of volumetric reconstruction is formulated as a non-linear least-squares problem and solved explicitly without the use of neural networks. This enables the use of solvers with a higher rate of convergence than what is typically used for neural networks, and fewer iterations are required until convergence. The volume is represented using a grid of voxels, with the scene surrounded by a hierarchy of environment maps. This makes it possible to get clean reconstructions of 360\ub0 scenes where the foreground and background is separated. A number of synthetic and real scenes from well-known benchmark-suites are successfully reconstructed with quality on par with state-of-the-art methods, but at significantly reduced reconstruction times

A low-cost, practical acquisition and rendering pipeline for real-time free-viewpoint video communication

We present a semiautomatic real-time pipeline for capturing and rendering free-viewpoint video using passive stereo matching. The pipeline is simple and achieves agreeable quality in real time on a system of commodity web cameras and a single desktop computer. We suggest an automatic algorithm to compute a constrained search space for an efficient and robust hierarchical stereo reconstruction algorithm. Due to our fast reconstruction times, we can eliminate the need for an expensive global surface reconstruction with a combination of high coverage and aggressive filtering. Finally, we employ a novel color weighting scheme that generates credible new viewpoints without noticeable seams, while keeping the computational complexity low. The simplicity and low cost of the system make it an accessible and more practical alternative for many applications compared to previous methods

PERF: Performant, Explicit Radiance Fields

We present a novel way of approaching image-based 3D reconstruction based on radiance fields. The problem of volumetric reconstruction is formulated as a non-linear least-squares problem and solved explicitly without the use of neural networks. This enables the use of solvers with a higher rate of convergence than what is typically used for neural networks, and fewer iterations are required until convergence. The volume is represented using a grid of voxels, with the scene surrounded by a hierarchy of environment maps. This makes it possible to get clean reconstructions of 360\ub0 scenes where the foreground and background is separated. A number of synthetic and real scenes from well-known benchmark-suites are successfully reconstructed with quality on par with state-of-the-art methods, but at significantly reduced reconstruction times

Real-Time Hair Filtering with Convolutional Neural Networks

Rendering of realistic-looking hair is in general still too costly to do in real-time applications, from simulating the physics to rendering the fine details required for it to look natural, including self-shadowing.We show how an autoencoder network, that can be evaluated in real time, can be trained to filter an image of few stochastic samples, including self-shadowing, to produce a much more detailed image that takes into account real hair thickness and transparency

Exploiting coherence in time-varying voxel data

We encode time-varying voxel data for efficient storage and streaming. We store the equivalent of a separate sparse voxel octree for each frame, but utilize both spatial and temporal coherence to reduce the amount of memory needed. We represent the time-varying voxel data in a single directed acyclic graph with one root per time step. In this graph, we avoid storing identical regions by keeping one unique instance and pointing to that from several parents. We further reduce the memory consumption of the graph by minimizing the number of bits per pointer and encoding the result into a dense bitstream

Photon Splatting Using a View-Sample Cluster Hierarchy

Splatting photons onto primary view samples, rather than gathering from a photon acceleration structure, can be a more efficient approach to evaluating the photon-density estimate in interactive applications, where the number of photons is often low compared to the number of view samples. Most photon splatting approaches struggle with large photon radii or high resolutions due to overdraw and insufficient culling. In this paper, we show how dynamic real-time diffuse interreflection can be achieved by using a full 3D acceleration structure built over the view samples and then splatting photons onto the view samples by traversing this data structure. Full dynamic lighting and scenes are possible by tracing and splatting photons, and rebuilding the acceleration structure every frame. We show that the number of view-sample/photon tests can be significantly reduced and suggest further culling techniques based on the normal cone of each node in the hierarchy. Finally, we present an approximate variant of our algorithm where photon traversal is stopped at a fixed level of our hierarchy, and the incoming radiance is accumulated per node and direction, rather than per view sample. This improves performance significantly with little visible degradation of quality

Efficient rendering of semi-transparent geometry in real-time

Rendering images of semi-transparent geometry representing e.g. windows, transparentcloth or architectural models made transparent for visualization purposes presentsa difficult problem in interactive applications due to the limitation of the ubiquitous zbufferof only storing one object per pixel, whereas correctly resolving the color oftransparent geometry requires considering the color of all fragments within a pixel inthe correct order. Very fine geometry, representing e.g. hair, fur or foliage that has trianglesto thin to be correctly captured by anti-aliasing hardware must often be treatedas semi-transparent and so suffer from the same limitations. Casting shadows from eitherof these types of geometry presents an analogous problem as standard real-timeshadowing techniques do not take into account the opacity of materials.This thesis presents a number of novel solutions to the problem of rendering andcasting shadows from semi-transparent and fine geometry. First, we present a methodfor simply sorting camera-facing billboards efficiently on the GPU. Second, the specificproblem of rendering alpha-composited self-shadowing hair or fur, where the geometryis typically a large number of short line-segments is addressed. Third, we present amethod that handles any type of transparent geometry under the assumption that alphacan be considered constant for all fragments and finally, we present a general algorithmthat is able on current hardware to render any type of fine or transparent geometry whichis equivalent to backwards Monte Carlo raytracing (with no changes in ray direction).For each new method introduced, comparisons to previous work are made consideringperformance as well as quality of the rendered images

Efficient rendering of semi-transparent geometry in real-time

Rendering images of semi-transparent geometry representing e.g. windows, transparentcloth or architectural models made transparent for visualization purposes presentsa difficult problem in interactive applications due to the limitation of the ubiquitous zbufferof only storing one object per pixel, whereas correctly resolving the color oftransparent geometry requires considering the color of all fragments within a pixel inthe correct order. Very fine geometry, representing e.g. hair, fur or foliage that has trianglesto thin to be correctly captured by anti-aliasing hardware must often be treatedas semi-transparent and so suffer from the same limitations. Casting shadows from eitherof these types of geometry presents an analogous problem as standard real-timeshadowing techniques do not take into account the opacity of materials.This thesis presents a number of novel solutions to the problem of rendering andcasting shadows from semi-transparent and fine geometry. First, we present a methodfor simply sorting camera-facing billboards efficiently on the GPU. Second, the specificproblem of rendering alpha-composited self-shadowing hair or fur, where the geometryis typically a large number of short line-segments is addressed. Third, we present amethod that handles any type of transparent geometry under the assumption that alphacan be considered constant for all fragments and finally, we present a general algorithmthat is able on current hardware to render any type of fine or transparent geometry whichis equivalent to backwards Monte Carlo raytracing (with no changes in ray direction).For each new method introduced, comparisons to previous work are made consideringperformance as well as quality of the rendered images